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(57) A composite camera system includes a control 
section for performing positional control and magnifica- 
tion ratio control of at least one zoom camera for omni- 
directional image data centered on a prescribed portion 
thereof, the omnidirectional image data being obtained 



by an omnidirectional camera capable of taking an on> 
nidirectional image over a viewing angle of a maximum 
of 360 degrees; and a display section for displaying an 
omnidirectional image taken by the omnidirectional 
camera and a zoom image taken by the 200m camera. 
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Description 

BACKGROUND OF THE INVENTION 

1 . FIELD OF THE INVENTION: 

[0001] The present Invention relates to a composite 
camera system for taking an omnidirectional image us- 
ing an omnidirectional camera capable of capturing in- 
formation of a viewing angle of a maximum of 360 de- 
grees, and zoom-photographing an area including an 
image area which is automatically or manually specified 
from the omnidirectional image by a zoom camera; a 
zoom camera Image display control method and a zoom 
camera control method using the same/ a control pro- 
gram usable for the same. and a computer readable re- 
cording medium having the program storied therein. 

2. DESCRIPTION OF THE RELAT ED ART . 

[0002] Monitoring cameras conventionally and gener- 
ally known as used in security devices installed in for 
example, financial institutions and supermarkets in- 
clude fixed-type and pivotablb cameras. Usually, a man- 
ager checks a video taken by such a monitoring camera 
using a monitoring device or the like in the manager's 
office, so that the manager can recognize any abnor- 
mality caused by, for example, a trespasser or a fire and 
take emergency measures. . * 
[0003] A monitoring instrument for allowing the man- 
ager to monitor a video taken by the monitoring camera 
on a screen of a monitoring device is inevitably large- 
sized and can be quite expensive. Even such a large 
instrument is still insufficient for accurately monitoring a 
target of crime prevention at a remote location. 
[0004] Generally known security monitoring devices 
include devices which issue an alarm using (i) sensors 
for sensing a trespasser who intercepts , for example, 
laser light or infrared rays; or (ii) sensors for sensing 
fires and gas leaks. When such a security device using 
such a sensor gives an alarm, the manager can recog- 
nize that an abnormality has occurred , but cannot obtain 
accurate information by visually checking the degree or 
details of the abnormality. . 

[0005] As the number of crimes increases year by 
year, a security monitoring system has been demanded 
for allowing the manager at a remote location to accu- 
rately identify the occurrence of an abnormality while 
monitoring the target of crime prevention without requir-" 
ing the manager to pay special attention. 
[0006] Proposals for solving the abovendescribed 
problems are described in, for example, Japanese Laid- 
Open Publication No. 3-84700 directed to "Remote 
Monitoring Device", Japanese Laid-Open Publication 
No. 4-176484 directed to "Monitoring System in Play 
Hail", and Japanese Laid-Open Publication No. 
11-259775 directed to "Monitoring System". 
[0007] The "Remote Monitoring Device" disclosed by 



Japanese Laid-Open Publication No. 3-84700 operates 
as follows; A plurality of sensors are dispersed in blocks 
of a monitoring area such that each sensor can detect 
. an abnormality in the respective block. When an abnor- 

5 mality is detected, aTV camera is automatically directed 
to the site of the abnormality, and transfers the abnor- 
mality detection signal and the video signal captured by 
the TV camera to a terminal in the monitoring office. The 
terminal automatically displays the image of the site of 

10 the abnormality and related information. Thus, the situ- 
ation at the site of the abnormality is recognized easily 
and with certainty. 

[0008] the "Monitoring System in Play Hail" disclosed 
. by Japanese Laid-Open Publication No. 4-176484 op- 
*5 erates as follows. A monitoring video camera Is set to a 
rnonitoring state for monitoring a game machine, which 
has been determined to be in an abnormality based on 
the result of analysis of Information obtained by abnor- 
mality detection means. The video information on that 
20 game machine is displayed on the screen of the video 
camera. Thus, the monitoring efficiency of each game 
machine can be > improved at low equipment cost. 
[0009] The "Monitoring System" disclosed by Japa- 
nese Laid-Open Publication No. 11-259775 operates as 
25 follows. An image of a monitoring area is captured by a 
wide-range camera, and the captured image is com- 
pared with an image of a normal state so as to find a 
difference. Thus, ah abnormality is detected. When an 
abnormality occurs, an image of the abnormality is cap- 
30 tured by the wide-range camera, and also an image at 
an important location is captured by a zoom camera. 
The Image obtained by the wide-range camera and the 
image obtained by the zoom camera are transferred to 
a terminal computer. Thus, a manager at a remote lo- 
ss cation can recognize thatthe situation of the abnormality 
relatively in detail, and accurately specify the degree of 
the abnormality and the like. 

[0010] The above-described technologies have the 
following problems. 
40 [0011] In the case of the "Remote Monitoring Device" 
disclosed by Japanese . Laid-Open Publication No. 

3- 84700, the sensors are dispersed. This means that 
the sensors can detect abnormalities only at sites where 

. the sensore are located. in addition, certain types of sen- 

6 sor can only detect an abnormality of the respective 
type. There are other problems in that the reliability of 
the sensors dispersed may be lowered, and it is difficult 
to control the position of the TV camera with respect to 
the sensors. 

50 [0012] In the case of the "Monitoring System in Play 
s Hall" disclosed by Japanese Laid-Open Publication No. 

4- 176484, abnormality detection means including vari- 
ous sensors needs to be located in the vicinity of each 
game machine; Therefore, abnormalities only at sites 

55 where the sensors are located can be detected. There 
are other problems in that the sensors may malfunction 
because of the environment in which the sensors are 
located, and it is troublesome to control the position of 
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the TV camera with respect to the various sensors. 
[0013] In the case of the "Monitoring System" dis- 
closed by . Japanese Laid-Open Publication No. 
1 1 -259775/rthe wide-range camera acts as- one sensor 
for comparing an Image of a target of monitoring and an 
image of a normal state and detectingan abnormality 
based on the difference. The zoom camera Is set to be 
directed to several preset important locations. The zoom 
camera is not structured so as to track the sites of ab- 
normality photographed by the wide-range camera and 
to have the position and the magnification ratio thereof 
controlled. Therefore, when there are many important 
locations, the important location at which an abnormality 
has occurred needs to be visually checked. This re- 
quires images of a plurality of Important locations to be 
taken simultaneously. 

SUMMARY OF THE INVENTION : 

[0014] According to one aspect of the Invention, a 
composite camera system includes a control section for 
performing positional control and magnification ratio 
control of at least one zoom camera ratio for omnidirec- 
tional image data centered on a proscribed portion 
thereof, the omnidirectional image data being obtained 
by an omnidirectional camera capable of taking an om- 
nidirectional image . over a viewing angle of a maximum 
of 360 degrees; and a display section for displaying an 
omnidirectional image taken by the omnidirectional 
camera and a zoom image taken by the zoom camera, 
[0015] : In one embodiment of the invention, the posi- 
tional control for controlling a photographic position is 
performed by pan driving and tilt driving, and the mag- 
nification ratio control is performed by zoom driving. 
[0016] In one embodiment of the invention, the pre- 
scribed portion of the- omnidirectional image data is a 
divided image area which is set In a photographed im- 
age. ' = •• .,. 
[0017] In one embodiment of the invention, the control 
section includes a dividing section for dividing the om- 
nidirectional image displayed on a display screen of the 
display section into a plurality of image areas; a speci- 
fying section for automatically or manually specifying at 
least one image area from the plurality of image areas; 
a converting section for converting Information on the at 
least one image area specif ied by the specifying section 
into control information for controlling the zoom camera 
to be at a prescribed position and to have a prescribed 
magnification ratio for photographing ah image area 
specified by the specifying section; and a zoom camera 
control section for. controlling the zoom camera to be at 
the prescribed position and to have the prescribed mag- 
nification ratio for photographing the image area speci- 
fied by the specify ing section . based on the control in- 
formation converted by the converting section. 
[0018] In one embodiment of the invention, the omni- 
directional camera Includes an .optical system for pro- 
jecting the omnidirectional image using a mirror of a par- 



abolic or hyperbolic convex body of revolution, or a fish- 
eye lens. 

[0019] In one embodiment of theinventlon, the com- 
posite camera system further includes a control Infor- 

5 matlon setting section for presetting the control informa- 
tion for controlling the zoom camera to be at the pre- 
scribed position and to have the prescribed magnifica- 
tion ratio for each of the plurality of image areas obtained 
by the dividing section; and a control information table 

10 for storing the preset control information in correspond- 
ence with the position of each of the plurality of image 
areas. The converting section obtains the control infor- 
mation for the image area specified by the specifying 
section from the control information table and/or from 

15 calculation using a prescribed calculation procedure. 
[0020] In one embodiment of the invention, the com- 
posite camera system further includes an Image mem- 
ory section for temporarily storing the omnidirectional 
image taken by the omnidirectional camera frame by 

2° frame sequentially ; and a moving body detection section 
for detecting a movement of a moving body by perform- 
. ing pattern matching of successive frames of the omni- 
directional image stored in the image memory section. 
The detected moving body is zoom-photographed by 

2S the zoom camera, and an en larged image of the moving 
body is displayed oh the display screen of the display 
■ section..: 

[0021] In one embodiment of the invention, when the 
omnidirectional image is circular, the dividing section di- 
30 vides the omnidirectional image concentrically and ra- 
dially, and displays the plurality of image areas of the 
omnidirectional image together with area frames defin- 
ing the plurality of image areas.- • 
[0022] in one embodiment of the invention, the divid- 
es ing section displays the plurality of image areas of the 
omnidirectional image together with area frames and ar- 
ea numbers defining the plurality of Image areas . The 
control information setting section presets the control in- 
formation for controlling the zoom camera to be at the 
40 prescribed position and to have the prescribed magnifi- 
cation ratio for each of the plurality of image areas pro- 
vided with the area numbers. . 
[0023] In one embodiment of the invention, the com- 
posite camera system further includes an invalid area 
*5 setting section for presetting, as invalid, an arbitrary ar^ 
ea of the omnidirectional Image which is unstable due 
to environmental influence. The Invalid area setting sec- 
tion invalidates the specification of the image area by 
the specifying section. 
so [0024] In one embodiment of the invention, the com- 
posite camera system further includes a monitoring area 
sotting section for setting at least one monitoring area 
in the omnidirectional image; and an alarm information 
output section for outputting alarm information when the 
55 moving body detecting section detects that the moving 
body has entered the monitoring area. The zoom cam- 
era zoom-photographs the moving body which has en- 
tered the monitoring area and displays the photo- 
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graphed image of the moving body on the display screen 
of the display section. 

[0025] In one embodiment of the invention, the com- 
posite camera system further includes a remote control- 
ler connected to the omnidirectional camera and the 5 
zoom camera via a wireless or wired communication 
line; An omnidirectional image taken by the omnidirec- 
tional camera and an image taken by the zoom camera 
are transferred to the remote controller via the commu- 
nication line. . : *o 
[0026] In one embodiment of the invention, the re^ 
mote controller includes at least the dividing section, the 
specifying section, and the converting section. 
[0027] According to another aspect of the invention, 
a composite camera image display control method in- ^ 
eludes the processing steps of performing positional 
control and magnification ratio control of at least one 
zoom camera for omnidirectional image data centered 
on a prescribed portion thereof, the omnidirectional im- 
age data being obtained by an omnidirectional camera %> 
capable of taking an omnicfirectional image over a view- 
ing angle of a maximum of 360 degrees; and displaying 
an omnidirectional image taken by the . omnidirectional 
camera and a zoom image taken by tno zoom camera. 
[0028] According to still another aspect of the inven- 25 
tion, a zoom camera control method for performing po- 
sitional control and magnification ratio control of at least 
one zoom camera for omnidirectional image data cen- 
tered on a prescribed portion thereof is provided. The 
omnidirectional image data is obtained by an omnidiree- 30 
tional camera capable of taking ah omnidirectional im- 
age over a viewing angle of a maximum of 360 degrees. 
The method includes the steps of displaying an omnidi- 
rectional image taken by the omnidirectional camera; di- 
viding the omnidirectional image displayed on a display & 
screen of the display section- into a plurality of image 
areas ; automatically or manually specifying at least one 
image area from the plurality of image areas; converting 
information on the at least one image area specif iedby 
the specifying section into control information for con- 40 
trolling the zoom camera to be at a prescribed position 
and to have a prescribed magnification ratio for photo- 
graphing the im age area specified by the specifying sec- 
tion; and controlling the zoom camera to be at the pre- 
scribed position and to have the prescribed magnified- 45 
tion ratio for photographing the Image area specified by 
the specifying section; based on thecbntrol information 
converted by the converting section, 
[0029] According to still another aspect of the inven- 
tion, a control program is provided for, causing a corn- so 
puter to execute the steps of displaying an omnidirec- 
tional imago taken by tho omnidirectional camera; divid- 
ing the omnidirectional image displayed on a display 
screen of the display section into a plurality of image 
areas; automatically or manually specifying at least one ; ss 
image area from the plurality of image areas; converting 
information on the at least one image area specified by 
trie-specifying section into control Information for con- 



trolling the zoom camera to be at a prescribed position 
and to have a prescribed magnification ratio for photo- 
graphing the image area specified by the specifying sec- 
tion; and controlling the zoom camera to be at the pre- 
scribed position and to have the prescribed magnifica- 
tion ratio for photographing the image area specified by 
the specifying section, based on the control information 
converted by the converting section. 
[0030] According to still another aspect of the inven- 
tion, a computer readable recording medium having the 
above described control program stored therein is pro- 
vided... 

[0031] According to still another aspect of the inven- 
tion, a control program for causing a computer to exe- 
cute the processing steps of the above-described com- 
posite camera image display control method is provid- 
ed. 

[0032] As described above, according to the present 
invention, posilional control and magnification ratio con- 
trol of the zoom camera are performed centered on a 
prescribed portion of omnidirectional image data, and 
the omnidirectional image and the zoom Image are dis- 
played. The omnidirectiorial camera is used to photo- 
graph a piuraiity of important locations Sim Urtafioousiy 
as included in the omnidirectional image, and the zoom 
camera is used to zoom-photograph a specified image 
area in the omnidirectional image. By specifying a pre- 
scribed image area in the omnidirectional image, the po- 
sition of the zoom camera can be easily set. 
[0033] The omnidirectional image over a viewing an- 
gle of a maximum of 360 degrees taken by the omnidi- 
rectional camera is divided into a plural ar- 
eas,: and an image area of interest is automatically or 
manually specified among the plurality of image areas. 
Thus, the specified image area of interest is zoom-pho- 
togfaphed-by the zoom carnera which can be driven to 
pan, tilt and zoom. Since a large-area monitoring area 
is photographed by one omnidirectional camera, it is not 
necessary to -install a plurality of monitoring sensors , 
for example; a plurality of detecting sensors or Inputsen- 
sors for automatic specification of the target image area. 
[0034] The plurality of image areas obtained by divid- 
ing the omnidirectional image act as a plurality of detec- 
tion sensors or input sensors to calculate the control in- 
formation for the Image area automatically or manually 
specified. Based on the control information, the zoom 
camera is controlled to be at a prescribed position (to 
assume a prescribed photographing direction) and to 
have a prescribed magnification ratio. 
[0035] The zoom camera can track a moving body in- 
cluded in the omnidirectional image taken by the omni- 
directional camera,- zoom-photograph the moving body, 
and display an enlarged image of the moving body on 
a display screen of the display section. 
[0036] In the casewhere a circular omnidirectional im- 
age is divided into a plurality of image areas, and control 
Information is set for controlling the zoom camera to be 
at a prescribed position and to have a prescribed mag- 
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nrfication ratio for each of the plurality of image areas, 
the division can be performed effectively and the image 
areas can be displayed so as to be easily identifiable. 
[0037] Control Inf ormatiori for : controlling the zoom 
camera to be at a prescribed positioned to have a pre- 
scribed magnification ratio for each of the plurality of im- 
age areas provided with the respective area numbers 
can be preset, while viewing the omnidrectional image 
displayed on the display screen of the display section. 
[0038] An image area, in the omnidirectional image, 
which is unstable due to environmental inf iuence can be 
invalidated in advance. Therefore, a moving body or an 
abnormality can be accurately detected automatically 
from the omnidirectional image, so that the position and 
the magnification ratio otthe zoom camera is accurately 
controlled forthe detected moving body or the site of the 
abnormality. 

[0039] A plurality of monitoring areas which are to be 
photographed in different photographing directions can 
be set for one omnidirectional camera. When a moving 
body is detected to have entered the monitoring area, 
alarm information including an alarm sound is output 
and also the zoom camera is zoomed to the moving 
body to display inc . imago of the moving body en the 
display screen of the display section - 
[0040] The omnidirectional camera and the zoom 
camera can be remote-controlled via a communication 
line. Therefore, at a site where a computer is Installed, 
the above-described division of the omnidirectional im- 
age, preset of the control information for controlling the 
position and the magnification ratio of the zoom camera 
for a specified image area, and photographing and dis- 
play of an area including the specified image area can 
be performed, .- 
[0041] Thus, the invention described herein makes 
possible the advantages of providing a composite cam- 
era system for photographing a plurality of important lo- 
cations simultaneously as included in an omnidirection- 
al image using an omnidirectional camera, and zoom- 
photographing a specified image area in the omnidirec- 
tional image with the position of a zoom camera being 
easily controlled; a zoom camera image display control 
method and a zoom camera control method using the 
same; a control program usable for the same; and a 
computer readable recording medium having the pro- 
gram stored therein. .... ■ 

[0042] These and other advantages of the present in- 
vention will become apparent to those skilled in the art 
upon reading and understanding the following detailed 
description with reference to the accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0043] 

Figure 1 is a schematic view illustrating a basic 
structure of a composite camera system according 
to an example of the present invention; 



. Figure 2 shows an example of a monitoring system 
. . using the composite camera system shown in Bg- 
- urel; ■ 

5 Figure 3 is a. block diagram illustrating a partial 
... structure of the composite camera system shown in 
Figure 1; . • 

Figure 4 is a perspective view illustrating a partial 
10 structure of an optical system of an omnidirectional 
camera included in the composite camera system 
shown in Figure 1; 

, Figure-S shows Display Example (1) according to 
: is the example of the present invention, which is for 
setting control information; 

Figure 6 shows Display Example (2) according to 
the example of the present invention, which is for 
20 setting control information; 

Figure 7 shows Display Example (3) according to 
the example of the present invention, which is for 
setting control inf orrnation; 
25 : : i . 

. Figure 8 shows Display Example (4) according to 
: the example of the present invention, which is for 
detecting a moving body; 

30 . Figure 9 shows Display Example (5) according to 
the example of the present invention, which is for 
setting an invalid area; and 

FigurelO is a flowchart illustrating a basic operation 
35 procedure of the composite camera system shown 
in Figure 1. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS : 

40 - -- ..; •-. 

[0044] Hereinafter, the present invention will be. de- 
scribed by way of illustrative, but: non-limiting examples 
with reference to the accompanying drawings: 
[0045] . Figure 1 is a schematic view illustrating a basic 

45 structure of a composite camera system 10 according 
to an example of the present Invention. 
[0046] In Figure 1, the composite camera system 10 
includes a zoom camera 1; an omnidirectional camera 
2, a remote controller 3, a zoom camera control unit 4 

50 acting as a zoom camera control section, a monitor 5, 
and communication lines 6a through 6d for connecting 
these elements so that those elements can communi- 
cate with each other. 

[0047] The zoom camera 1 includes; for example, a 
55 zoom lens, an imaging section (photographing section) 
including CCDs, and a driving section for driving and 
controlling the zoom lens to pan (to move to a prescribed 
position in a horizontal direction), to tilt (to move to a 
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prescribed position in a vertical direction) and to zoom 
(to have, a prescribed magnification ratio). The zoom 
camera 1 also includes a communication section for 
transferring a control signal or an image signal via a 
wireless or wired line. Based on control information sent 
from the zoom camera control unit 4, the zoom camera 
1 is driven and controlled to be a prescribed position (by 
pan end tilt) and to have a prescribed magnification ratio 
(by zoom). For example: 

zoom ratio: optical zoom: X23; 

electronic zoom: X10 

zoom speed: MAX1.6 seconds (at the preset time) 
number of presets: 255.(255 positions 
and magnification ratios) 

[0048] The omnidirectional camera 2 Includes, for ex- 
ample, an optical system (e.g., a mirror of a convex body 
of revolution, or a fish-eye lens) for projecting an optical 
image of an area over a viewing angle of a maximum of 
360 degrees, and an imaging section (photographing 
section), including CCDs, for photographing the optical 
image as a circularomnidirectional image. For example, 
a detection area (monitoring area) can have a radius of 
15 meters from the omnidirectional camera 2. 
[0049] The remote controller 3 includes, for example, 
a personal computer which can communicate with other 
devices via a wireless or wired line: The remote control- 
ler 3 includes an image area dividing section (dividing 
section) for dividing an omnidirectional image taken by 
the omnidirectional camera 2 into a- plurality of image 
areas; ah image area speeffying section (specifying sec- 
tion) for automatically or manually specifying at least 
one image area from the plurality of image areas ob- 
tained by the image area dividing section; a control in- 
formation converting section (converting section) for 
converting the information on the image area specified 
by thespecifying section ihtocohtrol information fbrcon- 
trolling the zoom camera 1 to move to a prescribed po- 
sition (by pan driving and tilt driving) and to have a pre- 
scribed magnification ratio (by zoom driving) for photo- 
graphing the image area specified by the image area 
specifying section; and a display section for displaying 
the omnidirectional image taken by the omnidirectional 
camera 2 on a display screen. 
[0050] The zoom camera control unit 4 includes a 
computer including, for example, a CPU, ROM, RAM, 
and I/O port. Based on the control information supplied 
from the remote controller 3, the zoom camera control 
unit 4 controls the position and the magnification ratio 
of the zoom camera 1 . The zoom camera control uhit 4 
has a function of communicating with other devices via 
a wireless or wired line. 

[0051] The monitor 5 Includes, for example, a liquid 
crystal display (LCD), a plasma display (PD), or ah elec- 
troluminescence display (ELD). The monitors displays 
an image which is zoom-photographed by the zoom 
camera 1. 



[0052] Among the communication lines 6a through 
6d, the communication line 6a communicates the omni- 
directional camera 2 and the remote controller 3 to each 
other by an NTSC system; The communication line 6a 
5 transfers an -omnidirectional image taken by the omni- 
directional camera 2 to the remote controller 3. The 
communication line 6b connects the remote controller 

3 and the zoom camera control unit 4 to each other by 
an RS-232C system. The communication line 6b 

10 trarisf ers , to the zoom camera control unit 4 $ control in- 
formation for controlling the zoom camera 1 to be at a 
prescribed; position (to assume a prescribed photo- 
graphing direction) and to have a prescribed magnifica- 
tion ratlo.Thecommunlcation line 6c connects the zoom 

« camera control unit 4 and the zoom camera 1 to each 
other by an RS-4B5 system, so that the zoom camera 
control unit 4 controls the position and the magnification 
ratio of the zoom camera 1 for zooming the specified 
image area. The communication line 6d connects the 

20 zoom camera 1 and the monitor 5 by an NTSC system. 
The communication line 6d transfers an Image zoom- 
photographed by the zoom camera 1 to the monitor 5. 
[0053] In this example, the zoom camera control unit 

4 is provided as a separate unit. Alternatively, the zoom 
25 camera control unit 4 may be incorporated into the re- 
mote controllers or the zoom camera 1 . The zoom cam- 
era 1 and the omnidirectional camera 2 can each have 
a function- of communicating with other devices via a 
wireless- or wired line. :. 

30 [0054] Figure 2 shows an exemplary monitoring sys- 
tem including the composite camera system 10 shown 
in Figure 1 . Identical elements previously discussed with 
respect to Figure 1 bear identical reference numerals 
and the detailed descriptions thereof will be omitted. 

35 [0055] In Figure 2, the composite camera system 10 
continuously monitors sites f or monitoring such as; for 
example, a schoolyard/school gate/gym/elevator en- 
trance, a parking lot, an office, a store, an apartment 
building, or a place of amusement, with the omnidirec- 

40 tional camera 2 over a wide range (viewing angle: 360 
degrees) with no dead angle. When detecting a tres- 
passer M, the zoom camera 1 is controlledto be directed 
7 to the trespasser M and zoom-photographs the tres- 
passer M at a prescribed magnification ratio; The image 

45 is displayed by the monitor 5>whlch is located separate- 
ly from the remote controller 3 but acts as a display sec- 
tion of the remote controller 3. An enlarged image of the 
trespasser M may be displayed both on a display screen 
of the remote controller 3 and a display screen of the 

so monitors.--. 

[0056] In this example, the zoom camera 1 and the 
omnidirectional camera 2 arc installed over the entrance 
to a school as shown in Figure 2. The trespasser M en- 
tering through the school gate is detected by the omni- 

55 directional camera 2 and is zoom-photographed by the 
zoom camera 1. An enlarged image of the trespasser M 
is displayed on the monitor 5 installed in a monitoring 
room: 
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[0057] Although not shown in Figure 2, the remote 
controller 3 for remote-controlling the omnidirectional 
camera 2, the zoom camera control unit 4 for controlling 
the position and the magnification ratio of the zoom cam- 
era 1, arid the communication lines 6a through 6d for 
communicating the remote controller 3, the zoom cam- 
era 1, the omnidirectional camera2 and the like are also 
installed in the monitoring room. 
[0058] Figured is a block diagram illustrating a partial 
structure of the composite camera system 10 shown in 
Figure 1. Identical elements previously discussed with 
respect to Figure 1 bear identical reference numerals 
and the detailed descriptions thereof will be omitted. 
[0059] In Figure 3, as described above, the composite 
camera system 10 includes the zoom camera 1 , the on*- 
nldirectional camera2, the remote controller 3, the zoom 
camera control unit 4 acting as a zoom camera control 
section, the monitors, and communication lines 6 (6a 
through 6d as shown in Figure 1). The composite cam- 
era system 1 0 further includes an alarm information out- 
put device 7. 

[0060] The zoom camera 1 includes , for example, an 
imaging section (photographing section) 1a; a driving 
section lb tor driving ana controlling the zoom iens to 
pan (to move to a prescribed position in a horizontal di- 
rection), to tilt (to move to a prescribed position in a ver- 
tical direction) and to zoom (to have a prescribed mag- 
nification ratio),; and a communication section 1c for 
communicating with other devices via a wireless or 
wired line for transference of control signals and image 
signals. Based on control Information sent from the 
zoom camera control unit 4 ■', the zoom camera 1 is con- 
trolled to have a prescribed position (to assume a pre^ 
scribed photographing direction) and to have a pre^ 
scribed magnification ratio. The driving section 1b in- 
cludes, for example, a pulse motor, a DC servo motor, 
anultrasohicmotor, or an encoder, and has a function of 
automatic high-speed three-dimensional control of the 
position and the magnification ratio: 
[0061J As shown In Figure 3, the omnidirectional cam- 
era 2 includes an optical system 2a, an imaging section 
(photographic section) 2b including CCDs, and a com- 
munication section 2c for communicating with other de- 
vices for transference of control signals and image sig- 
nals via a wireless or wired line. 
[0062] The optical system 2a Includes, for example, 
a parabolic or hyperbolic mirror of a convex body of rev- 
olution or a fish-eye lens, and a cylindrical ': or cup- 
shaped light-transmissive body for holding a CCD cam- 
era which is included in the imaging section 2b. The op- 
tical system 2a projects an Optical image over an omni- 
directional viewing angle of a maximum of 360 degrees. 
For example, the optical system 2a projects an optical 
image obtained by projecting (collecting) light reflected 
by the mirror of a convex body of revolution by a lens or 
a fish-eye lens. 

[0063] The imaging section 2b includes a CCD cam- 
era including, for example, an imaging lens, a CCD sec- 



tion, an A/D conversion circuit, and an image processing 
circuit. The imaging section 2b photographs the optical 
image projected by the optical system 2a as omnidirec- 
tional image data. 

5 [0064] The zoom camera control unit 4 includes a 
computer including, for example, a CPU, ROM, RAM, 
and I/O port. The zoom camera control unit 4 also has 
a function of communicating with other devices via a 
wireless or wired line: Based on the control information 

10 supplied from the remote controller 3, the zoom camera 
control unit 4 controls the position and the magnification 
ratio of the zoom camera 1 v 
[0065] The monitor 5 includes a liquid crystal display 
(LCD), a plasma display (PD), an electroluminescence 

w display (ELD), or various other display devices. The 
monitors also has a function of communicating with oth- 
er devices via a wireless or wired line. The monitor 5 
receives an enlarged Image which is zoom-photo- 
graphed by the zoom camera 1 from the remote contra I- 

20 jer 3 and displays the enlarged image on a display 
screen thereof. 

[0066] The communication lines 6 include a plurality 
of communication lines for transferring data signals by, 
for cxampio , an NT3C , RS-232C , or RS-4S5 system . 

25 The communication lines 6 may use a wi reless LAN for 
communicating specific short-distance aerial transmis- 
sion signals such as; for example I rDAControl-system 
infrared signals or Bluetooth-system radio wave signals. 
[0067] The alarm information output device 7 in- 

30 dudes, for example; a buzzer, speaker, and an alarm 
lamp. When, for example, a moving body such as a tres- 
passer M (Figure 2) enters a monitoring area, the alarm 
inform ation output device 7 outputs alarm information 
including an alarming sound and an alarming light. 

35 [0068] The remote controllers control sec- 

tion 3a, a program memory 3b, a buffer memory 3c, an 
image memory section 3d, a display section 3e, a con- 
trol information table 3f, an operation input device 3g, a 
communication section 3h, and a bus 31 for transferring 

40 data between these sections. 

[0069] The control section 3a Includes a CPU (central 
processing unit) of the computer. Based on a control 
program, the control section 3a controls the sections of 
the composite camera system 10. The detailed structure 

45 of the control section 3a will be described later. 

[0070] The program memory 3b includes, for exam- 
ple, a readable ; recording medium ^uch as a ROM, 
EPROM, EEPROM, Floppy® disc, optical disc (for ex- 
ample, compact disc), or hard disc. The program mem- 

50 ory 3b stores the control program for starting the control 
section 3a and causing the control section 3a to execute 
various operations. 

[0071] The buffer memory 3c Includes, for example, 
a readable recording medium such as a RAM, EPROM, 
55 EEPROM, Floppy® disc or hard disc. The buffer mem- 
ory 3c temporarily stores data which is being controlled. 
[0072] The image memory section 3d includes, for ex- 
ample, a readable recording medi urn such as a RAM , 
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EPROM,-EEPROM, Floppy® disc or hard disc. The im- 
age memory section 3d temporarily stores an omnidi- 
rectional image taken by the omnidirectional camera 2 
frame by frame sequentially. - 
[0073] ..The display, section 3e includes, for example, 
a liquid crystal display (LCD), a piasma display (PD); or 
an electroluminescence display (ELD)/The display sec- 
tion 3e displays an omnidirectional image taken by the 
omnidirectional camera 2 on a display screen thereof. 
[0074] The control information table 3f includes, for 
example^ a readable recording medium such as a RAM, 
EPROM, EEPROM, Floppy® disc or hard disc; The con- 
trol information table 3f stores control information (infor- 
mation on the position (photographic direction) and the 
magnification ratio of the zoom camera 1 ) , which is pre- 
set by a control information setting section 33a (de- 
scrlbedbelow) Included in the control section 3a, in cor- 
respondence with Information on each of the Image ar- 
eas (for example, the area number or the representative 
coordinate of each of the image areas). 
[0075] The operation input device 3g includes, for ex- 
ample, a mouse and a keyboard. :. 
[0076] : ;The communication section 3h includes, for 
example, an antenna, modem (signal modulation and 
demodulation device), a wireless signal conversion cir- 
cuit, or a communication line connection circuit. The 
communication section 3 h is communicable with the 
omnidirectional camera 2, the zoom camera control unft 
4^ the monitors, the alarm information output device 7 
and the like via the communication lines 6; 
[0077] The detailed structure of the control section 3a 
of the remote controller 3 will be described. The control 
section 3a includes a dividing section 31a, a specifying 
section 32a, the control information setting section 33a; 
a converting section 34a, a moving body detecting sec- 
tion 35a;an invalid area setting section 36a, a monitor- 
ing area setting section 37a, and a cursor control section 
36a. The control section 3a drives the zoom camera 1 
through the zoom camera control unit 4 so as to be at a 
prescribed position (by pan and tilt driving) and to have 
a prescribed magnification ratio (by zoom driving) for 
photographing the omnidirectional image data obtained 
by the omnidirectional camera 2, centered on a speci- 
fied portion thereof . . . \ - 
[0078] .: Based on a dividing program of the control pro- 
gram; the dividing section 31 a divides an omnidi rection- 
al image, for example, concentrically and radially. Then, 
the dividing section 31a displays the divided image ar- 
eas of the omnidirectional image, together with an area 
frame which clearly defines each; divided image area 
and ah area number added to each divided image area. 
[0079] Based on an operation specifying instruction 
from the operation input device 3g and a specifying pro- 
gram of the control program, the specifying section 32a 
automatically of manually specifies an image area (or a 
prescribed number of image areas) defined by an area 
frame (or a prescribed number of image frames) and an 
area number (or a prescribed number of area numbers), 



among the plurality of image areas displayed. 
[0080] Based on an operation setting instruction from 
the operation input device 3g and a control information 
setting program of the control program, the control in- 
5 formation setting section 33a presets control informa- 
tion for controlling the position and the magnification ra- 
tio of the zoom camera 1 on an image area-by-image 
area basis. 

[0081] Based on a conversion program of the control 

10 program, the converting section 34a converts informa- 
tion on the image area specified by the specifying sec- 
tion 32a into control information for controlling the posi- 
tion and the magnification ratio of the zoom camera 1 
for photographing theimage area specified by the spec- 

* 5 ifying section 32a. The converting section 34a may ob- 
tain the controLinformatlon for the specified Image area 
from the control information table 3f and/or from calcu- 
lation using a prescribed calculation procedure, based 
oh the conversion program. 

so [0082] : . Based oh a moving body detecting program of 
the control program, the moving body detecting section 
35a performs pattern matching of successive frames of 
the omnidirectional image which has been stored in the 
image memory section 3d, so as to create a difference 

25 image. Based on the difference image, the moving body 
detecting section 35a detects a movement of a moving 
body (Including, for example, an object which changes 
the image, such as a trespasser M shown in Figure 2). 
The detected moving body is zoom-photographed by 

30 the zoom camera 1 , and an enlarged image of the de- 
tected moving body is displayed on a display screen of 
the monitor 5 and/or the display section 3e. 
[0083] Based on an operation setting instruction from 
the operation input device 3g and an invalid area setting 

35 program of the control programme invalid area setting 
section 36a presets, as invalid, an arbitrary image area 
in the omnidirectional image which is unstable due to 
environmental influence. Theexpression "image area 
which is unstable : due to environmental Influence" 

40 : means an image area representing , : for example, an ob- 
ject which moves in. a horrhal state or an object which 
moves but. is too small to be a trespasser or anything 
causing an abnormality. The invalid area setting section 
: 36a can also display the image area which has been 

45 preset as Invalid on the display section 3e, so that the 
Image area is identified. When the image preset as 
invalid has been specified by the specifying section 32a, 
the invalid area setting section 36a can invalidate the 
specification;; 

bo [0084] Based on an operation setting Instruction from 
the operation input device 3g and a monitoring area set- 
ting program of the control program, the monitoring area 
setting section 37a sets at least one monitoring area in 
the omnidirectional area. Any image area other than the 

55 image area which has been invalidated by the invalid 
area setting section 36a may be set as a monitoring ar- 
' ea.- "' •• • ...v. 

[0085] Based on an operation instruction from the op- 
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eratiori input device 3g and a cursor control program of 
the control program, the cursor control section 38a con- 
trols the position of the cursor on the display screen. 
[0086] The composite camera system 10 in this ex- 
ample having the above^descrlbed structure; when ap- 
plied to a monitoring system, provides, for example, the 
following effects. .- - 

(1) The omnidirectional camera 2 realizes monitor- 
. ing over 360 degrees with no dead angle. ■ 

(2) A combination of the omnidirectional camera 2 
and the usual zoom camera 1 realizes high-speed 
display of a high-quality zoom-up image. 

(3) An automatic tracking function is provided of de- 
tecting a moving body by the omnidirectional cam- 
era 2 and zoom-photographing the moving body by 
the usual zoom camera 1. 

(4) The omnidirectional camera 2 realizes wide-ar- 
ea detection. 

(5) A manual tracking function is provided of zoom- 
photographing, at a high speed, an omnidirectional 
image, centered on a specified area thereof; 

[0087] The above-described control program is 
stored in prescribed readable recording memories in the 
remote controller 3 as follows. ■ 
[0088] The program memory 3b stores a display step 
of displaying an omnidirectional image taken by the om- 
nidirectional camera 2 on a display screen; a dividing 
step of dividing the omnidirectional image into a plurality 
of image areas; a specifying step of automatically or 
manually specifying at. least . one image area from the 
plurality of divided image areas; and a converting step 
of converting information on the specified image area 
into control information for contrail ing the zoom camera 
1 to be at a prescribed position and to have a prescribed 
magnification ratio for the specified image area. - ". 
[0089] : A memory section in the zoom camera control 
unit 4 stores a zoom camera control step for controlling 
the position and the magnification ratio of the zoom cam- 
era 1 for photographing the image area specified by the 
specifying section 32a based on the converted control 
; information. 

[0090] At least the above-mentioned steps are exe- 
cuted by the computer. The control program may also 
include, for example; a control information setting step, 
a moving body detecting step, an invalid area setting 
step, a monitoring area setting step, a designating step 
of designating the image area specified by each of these 
steps , an alarm information output control :step, and a 
zoom image display step. 

[0091] Figure 4 is a perspective view illustrating a par- 
tial structure of the optical system 2a (Figure 3) of the 
omnidirectional camera 2. . 



[0092] In Figure 4 , the optical system 2a includes a 
mirror of a convex body of revolution. When a hyperbolic 
curve Is rotated about axis Z, a hyperboloid of two 
sheets is obtained. The mirror of the convex body of rev- 
olution has a mirror surface on a convex surface of one 
of the hyperboloids (Z > 0). 

[0093] The one hyperboloid is represented by the fol- 
lowing expressions; 

•Y 2 )/a 2 }-Z 2 /b 2 = -1 

c 2 =(a 2 + b 2 ) 

[0094] Here, M a" and "b" are constants defining the 
shape of the hyperboloid. "c M is a constant defining the 
position of the focal point of the hyperboloid. These ex- 
pressions and constants are included in conversion in- 
. formation for converting an omnidirectional image into 
a panoramic image or a perspective image. For exam- 
ple, these expressions and constants are pre-stored in 
the program memory 3b in Figure 3. 
[0095] The mirror of a convex body of revolution has 
two focal points, i.e., a first focal point I and a second 
focal point II. Light entering from the outside of the . mir- 
ror and directed toward the first focal point I is reflected 
by the optical system 2a (i:e., by the mirror of the convex 
body of revolution) and is entirely directed to the second 
focal point II. 

[0096] The rotation axis of the mirror of a convex body 
of revolution is matched tothe optical axis of the camera 
lens of the imaging section 2b, and a first principal point 
of the camera lens is located at the focal point II. Owing 
to this arrangement, an image photographed by the im- 
aging section 2b has a center of view point at the first 
focal point I. When the image is viewed with the first focal 
point I as the center of view, the position of the viewpoint 
is not changed in accordance with the viewing direction. 
[0097] Figure 5 shows Display Example (1) according 
to this example of the present invention. Display Exam- 
ple (1 ) is for setting control information. 
[0098] Figure 5 shows a window 51 displayed on a 
display screen of the display section 3e. The window 51 
shows a circular omnidirectional image 52, and a pull- 
down menu 53; The pull-down menu 53 appears when 
the control information setting section 33a is started by 
an input operation performed using a mouse or a key- 
board. The. control information setting section 33a is 
started for dividing the omnidirectional image 52 into a 
plurality of image areas and for adding an area number 
(preset number) to each of the divided image areas. 
[0099] Figure 6 shows Display Example (2) according 
to this example of the present invention. Display Exam- 
ple (2) is for setting control information. ' 
[0100] Figure 6 shows a window 61 displayed on the 
display screen of the display Action 3e. The window 61 
shows a part of a circular omnidirectional image 62, and 
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a window 63. The window 63 appears when the control 
information setting section 33a is started by an input op- 
eration performed using a mouse or a keyboard. The 
control information setting section 33a is started for di- 
viding the-omnidirectional image 62 into .a plurality -of 
image areas and for adding an area number (preset 
number) to each of the divided image-areas. The win- 
dow 63 allows the user to input numerical figures which 
represent how many image areas the omnidirectional 
image 62 is to be equally divided into concentrically and 
radially. 

[0101] In this example, the omnidirectional image 62 
is divided into five concentric circles. The window 63 in 
Figure 6 shows the numbers of image areas included in 
the five concentric circles. The omnidirectional image 62 
after the division is shown in Figure-7; Here, the omni- 
directional image 62 is divided into 255 image areas. 
[0102] Flgure7 shows Display Example (3) according 
to this example of the present invention. Display Exam- 
ple (3) is for setting control information. 
[01 03] Figure 7 shows a window 71 displayed on the 
display screen of the display section 3e. The window 71 
shows a circular omnidirectional . Image 72, Image 
frames 73 obtained as a result of equally dividing the 
omnidirectional image 72 concentrically and radially by 
the dividing section 31a based on the numerical figures 
which was input in the window 63 (Figure 6), and area 
numbers 74 added to the divided image areas respec- 
tively.. In Figure 7, only some of the -area numbers are 
shown for the sake of simplicity, but actualiy area num- 
bers 1 through 255 are displayed in the respective im- 
age areas. • 

[0104] Control information for controlling the position 
and the magnification ratio of the zoom camera 1 is pro- 
vided in correspondence with each of the divided image 
areas numbered 1 through 255. In this example, the om- 
nidirectional image 72 is divlded into 255 image areas, 
and the zoom camera 1 can be zoomed to any image 
area r for example, the image area numbered 255! 
[01 05] Figure 8 shows Display Example (4) according 
to this example of the present invention. Display Exam- 
ple (4) is for detecting a moving body 
[0106] Figure 8 shows a window 81 displayed on the 
display screen of the display section 3e. The window 81 
shows apart of an omnidirectional Image 82 which is 
mostly hidden, difference image data 83, a vertical cur- 
sor line 84, a horizontal cursor line 85, arid an intersec- 
tion 86 of the vertical and horizontal cursor lines 84 and 
85. The difference image data 83 (white parts) is ob- 
tained by performing.pattern matching of successive 
frames of the omnidirectional image 8i2 which has been 
stored in the image memory section 3d. The difference 
image data .83 represents a part in which the image :has 
been changed due to, for example, a movement of a 
moving body. The vertical and horizontal cursor lines 84 
and 85 track the difference data based on a certain cri- 
terion (for example, whether the area of the white parts 
is a prescribed level or higher); so that the intersection 



86 represents the position of the moving body. By auto- 
matically specifying an area Including the position rep- 
resented by the intersection 86, the information on the 
image area including this position is converted intocon- 
5 trol information for zoom-photographing this image area 
bythe zoom camera 1. , 

[01 07] Figure 9 shows Display Example (5) according 
to this example of the present invention. Display Exam- 
ple (5) fe for setting an invalid area. 

10 [0108] Figure 9 shows a window 91 displayed on the 
display screen of the display section 3e. The window 91 
shows a circular ; omnidirectional image 92, image 
frames 93 obtained as a result of dividing the omnidi- 
rectional image 92 equally by the dividing section 31a 

is based ori the numerical figures which was input in the 
window 63 (Figure 6), and an: image area 94 which has 
been preset as Invalid by the invalid area setting section 
36a. As described above, the invalid area setting section 
36a presets, as invalid, an arbitrary image area in the 

20 omnidirectional image which is unstable due to the in- 
fluence of environment. The image area 94 preset as 
invalid bythe invalid area setting section 36a is not sub- 
jected to the specification by the specifying section 32a 
or the conversion by the converting section 34a. 

25 pi09] Figure 10 is a flowchart illustrating a basic op- 
eration procedure for the composite camera system 10 
shown in Figure 1. 

[0110] v As shown in Figure 10, an omnidirectional im- 
age is photographed in step S1 . More specifically, an 

30 omnidirectional image over a viewinjg angle of 360 de- 
grees around the omnidirectional camera 2 is taken by 
the omnidirectional camera 2. 
[0111] In step S2, the obtained omnidirectional image 
is displayed on the display screen of the display section 

35 3e; ;. 

[0112] In step S3, it is determined whether or not con- 
trol information for the omnidirectional image is to be 
set. When the control information is to be set (YES), the 
procedure goes to step S16; If the control information is 

40 not to be set (NO), the procedure goes to step S4. 
[0113] . In step S4; it is determined whether or not an 
image area in the omnidirectional Image is to be zoorn- 
photographed by automatic specification. When an im- 
age area in the omnidirectional image is to be zoom- 

45 photographed by automatic specification (YES), the 
procedure goes to step S5. When ho image area in the 
omnidirectional image is to be zoom-photographed by 
automatic specification, (NO), the procedure goes to 
stepS15. 

bo [0114] In step S5, it is determined whether or not there 
is an image area which has been preset as invalid by 
the invalid area setting section 36a; If there is such an 
area (YES), the procedure goes to step S6. If there is 
no such area (NO), the procedure goes to step S7. 

55 [0115] In step S6, the omnidirectional image exclud- 
ing the image area preset as invalid is temporarily stored 
In th e image memory section 3d frame by frame sequen- 
tialry. Then, the procedure goes to step S8. 
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[0116] In step S7, since there is no image area preset 
as invalid, the entirety of the omnidirectional image is 
temporarily stored in the image memory section 3d 
frame by frame sequentially. 
[0117] In step SB^ it is determined whether or not there 
is difference image data- in the omnidirectional image, 
using the moving body detecting section 35a. The dif- 
ference image data Is image data which has been posi- 
tional^ offset between two successive frames because 
of a movement of a moving body. If there is such data 
(YES), it Is determined that amoving body has been de- 
tected, and the procedure goes to step S9. If there Is no 
such data (NO), it is determined that no moving body 
has been detected, and the procedure returns to step 
S6. In step S6, pattern matching is performed for suc- 
cessive frames of the omnidirectional imagewhich have 
been stored in the image memory section 3d, thereby 
creating a difference Image. The difference image rep- 
resents the positional offset of the image data caused 
by a movement of a moving body. Based on the differ- 
ence image, it is determined whether or not there is a 
movingbody. - 

[0118] In step S9, the specifying section 32a is used 
to cause tho cursor to track the imago data which is po- 
sitional^ offset by a prescribed level or greater, in order 
to indicate mainly in which, image areas of the omnidi- 
rectional image that difference image data exists. Then, 
the area numbers of the Image areas corresponding to 
the image data are sequentially specified. 
[011 9] In step S1 0, the converting section 34a is used 
to convert the information on the imagevarea specified 
by the specifying section32a (for example, the area 
number of the image area) into the preset control infor- 
mation; 

[0120] In step S11, the communication section 3h is 
used to send the converted control information to the 
zoom camera control unit 4 via one of the communica- 
tion lines 6. 

[0121] In step S12, the zoom camera unit 4 controls 
the position and the magnification ratio of the zoom cam- 
era 1 based on the received control information via one 
of the communication lines 6. ; ; 
[0122] In step S13, a target image area is zoom-pho- 
tographed by the zoom camera 1 . 
. [0123] In step S14, the zoom-photographed image is 
displayed on the display screen of the monitors. Thus, 
the procedure is completed; 

[0124] In step S15, when the omnidirectional image 
was not zoom-photographed (NO in step S4), the spec- 
ifying section 32a, which received an operation specifi- 
cation instruction through the mouse, is used to specify 
the area number of the imago area to be zoom-photo- 
graphed by the zoom camera 1. Then, the procedure 
jumps to step S1 0. . 

[0125] . In step S1 6, when the control information is to 
be set (YES in step S3), the dividing section 31a Is used 
to divide the omnidirectional image displayed on the dis- 
play section 3e concentrically and radially. The area 



frame of each of the divided image area in the omnidi- 
rectional image is displayed, and the area number is 
added to each of the divided image areas. 
[0126] . In step S17, after step S1 6, the control infor- 

5 mation setting section 33a is used to preset the control 
information for controlling the position and the magnifi- 
cation ratio of the zoom camera 1 for each image area 
provided with the respective area number., 
[0127] ■■- In step S 18, after step S1 7, the preset control 

to information is stored in the control Information table 3f. 
Then, the procedure goes to step S4. 
[0128] As described above; the composite camera 
system i 0 according to this example of the present in- 
vention includes the following: the zoom camera 1 ca- 
ts • pable of being driven to pan; tilt and zoom; an omnidi- 
rectional camera 2 including an optical system 2a for 
projecting an optical image over a viewing angle of a 
maximum of 360 degrees, and an Imaging section 2b 
for photographing the optical image as an omnidirec- 

20 tional image; a display section 3e for displaying the pho- 
tographed omnidirectional image; a dividing section 31a 
for dividing the displayed omnidirectional image into a 
plurality of image areas; a specifying section 32a for au- 
tomatically or manually specifying at loast ono imago ar- 

25 eafromthe pluralityof imageareas; a converting section 
34a for converting the information on the image area 
specified by the specifying section 32a Into control in- 
formation for controlling the zoom camera 1 to be at a 
prescribed position and to have a prescribed magnifica- 

30 tion ratio for photographing the specified image area; 
and a zoom camera control section 4 for controlling the 
position and the magnification ratio of the zoom camera 
1 for photographing the specified image area, based on 
the converted control information. Owing to such a 

35 structure, the composite camera system 10 can take an 
omnidirectional image including images of a plurality of 
important sites using the omnidirectional camera 2, and 
display the images of these important sites simultane- 
ously as included in the omnidirectional image. The 

*o composite camera system 1 0 can also zoom-photo- 
graph a desired, specified image area In the omnidirec- 
tional image using the zoom camera 1. 
[0129] in the above example, the remote controller 3 
Is connected to the omnidirectional camera 2 and the 

*3 2 oom camera 1 vla the wireless or wired communication 
lines 6; Alternatively, the remote controller 3 and the 
zoom camera control unit.4 may be combined into one 
unit, orthe remote controller 3 may be provided in the 
vicinity of the omnidirectional camera 2 and the zoom 

so camera 1 as a controller. 

[01 30] In the above example, the composite camera 
system 10 includes one zoom camera 2. The present 
invention is not limited to such a structure. The control 
section 3a of the remote controller 3 may control the po- 

55 sltion and the magnification ratio of a plurality of zoom 
cameras via the zoom camera control unit 4. in this 
case, a plurality of display screens of the display section 
3e may be provided in correspondence with the plurality 
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of 700m cameras, or one display screen may display 
Images corresponding to the plurality of zoom cameras 
sequentially. 

[01 31] In the above example, the position of the zoom 
camera is controlled by pan driving and tilt driving/and 
the magnification of the zoom camera Is controlled by 
zoom driving. The pan driving, the tilt drivings and the 
zoom driving are merely examples for controlling the po- 
sition and magnification ratio; and the present invention 
is not limited to this. ■ . 

[0132] As described above, according to the present 
invention, positional control and magnification ratio coh- 
trolof the zoom camera are performed centered on a 
prescribed portion of omnidirectional Image data, and 
the omnidirectional image and the zoom image are dis- 
played. The omnidirectional camera Is used to photo- 
graph a plurality of important locations simultaneously 
as Included in the omnidirectional image, and the zoom 
camera is used to zoom-photograph a specified image 
area in the omnidirectional image. By specifying a pre- 
scribed image area in the omnidirectional image, the po- 
sition of the; zoom camera can be easi ly set. 
[0133] The omnidirectional image over a viewing an- 
gle of a maximum of 360 degrees taken by the omnidi- 
rectional camera Is divided into a plurality of image ar- 
eas, and an Image area of interest is automatically or 
manually specified among the plurality of Image areas. 
Thus, the specified image area of interest is zoom-pho- 
tographed by the zoom camera which can be driven to 
pan, tilt and zoom. Since a large-area monitoring area 
is photographed by one omnidirectional camera, It Is not 
necessary to install a plurality of monitoring sensors, for 
example,, a plurality of detecting sensors or input sen- 
sors forautpmatic specification of the target Image area. 
[0134] The plurality of Image areas obtained by divid- 
ing the omnidirectional image act as a plurality of detec- 
tion sensors or input sensors to calculate the control in- 
formation for the image area automatically or manually 
specified. Based on the control information, the zoom 
camera is controlled to be at a prescribed position (to 
assume a prescribed photographing direction) and to 
have a prescribed magnification ratio;; 
[0135] The zoom camera can track a moving body in- 
cluded in the omnidirectional image taken by the omni- 
directional camera, zoom-photograph the moving body, 
and display an enlarged image of the moving body on 
a display screen of the display section. 
[0136] In the case where a circular omnidirectional im- 
age is divided Into a plurality of image areas, and control 
information Is set for controlling the zoom camera to be so 
at a prescribed position and to have a prescribed mag- 
nification ratio for each of the plurality of image areas, 
the division can be performed effectively and the image 
areas can be displayed so as to be easily identifiable. 
[0137]' Control information for controlling the zoom 55 
camera to be at a prescribed position and to have a pre- 
scribed magnification ratio for each of the plurality of im- 
age areas : provided with the respective area numbers 



can be preset, while viewing the omnidirectional image 
displayed on the display screen of the display section. 
[0138] An image area/iri the omnidirectional Image, 
which is unstable due to environmental influence can be 
s invalidated In advance; Therefore, a moving body or an 
abnormality can be accurately detected automatically 
from the omnidirectional image, . so that the position and 
the magnification ratio of the zoom camera is accurately 
controlled for the detected moving body or the site of the 
10 abnormality. 

[0139] A plurality of monitoring areas which are to be 
photographed in different photographing directions" can 
be set for one omnidirectional camera. When a moving 
body is detected to have entered the monitoring area, 
15 alarm information including an alarm sound Is output 
and also the zoom camera is zoomed to the moving 
body to display the image of the moving body on the 
display screen of the display section; 
[0140] The omnidirectional camera and the zoom 
20 camera can be remote-controlled via a communication 
line. Therefore, at a site where a computer is installed, 
the above-described division of the omnidirectional im- 
age, preset of the control information for controlling the 
position and the magnification ratio of iho zoom camera 
25 for a specified image area,, and photographing and dis- 
play of an area including the specified image area can 
be performed. 

[0141 ] Various other modifications will be apparent to 
and can be readily made by those skilled in the art with- 
30 out departing from the scope and spirit of this invention. 
Accordingly, It is not intended that the scope of the 
claims appended hereto be limited to the description as 
set forth herein, but rather that the claims be broadly 
construed. 
35 : • . . ■ ■ 

Claims 

.1 . A composite camera system , comprising: 

■ . - - - -- a control section for performing positional con- 
. trol and magnification ratio control of at least 
one zoom camera ratio for omnidirectional im- 
age data centered on a prescribed portion 
45 -.- ; thereof, the omnidirectional Image data being 
. obtained by an omnidirectional camera capable 
of taking an omnidirectional image over a view- 
ing angle- of a maximum of 360 degrees; and 
a display section for displaying an omnidirec- 
tional image taken by the omnidirectional earn- 
era and a zoom image taken by the zoom cam- 
era. • • ■' 

2. A composite camera system according to claim 1 , 
wherein the positional control for controlling a pho- 
tographic position is performed by pan driving and 
. ■ tilt driving, and the magnification ratio control is per- 
formed by zoom driving; 
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3. A composite camera system according to claim 1 , 
whefeinthe prescribed portion of the omnidirection- 
al image data is a divided image area which is set 
in a photographed image. 

4. A composite camera system according to claim 1 , 
wherein the control section includes; 

a dividing section for dividing the omnidirection- 
al image displayed on a display screen of the 
display section into a plurality of image areas; 
a specifying section for automatically or manu- 
ally specifying at least one image area from the 
plurality of image areas; 
• a converting section for converting information 
on the at least one image area specif ied by the 
specifying section Into control Information for 
controlling the zoom camera to be at a pre- 
scribed position and to have a prescribed mag- 
nification ratio for photographing an image area 
specified by the . specifying section; and 
a zoom camera control section for controlling 
the zoom camera to be at the prescribed posi- 
tion and to have the prescribed magnification 
. ratio for photographing the image area speci- 
fied by the specifying section, .based on the 
control information converted by the converting 
section. 

5. A composite camera system according to claim 1, 
wherein the omnidirectional camera Includes an op- 
tical system for projecting the omnidirectional im- 
age using a mirror of a parabolic or hyperbolic con- 
vex body of revolution, or a fish-eye lens. 

6. A composite camera system according to claim 4, 
further comprising: , 

a control information setting section for preset- 
ting the control information for controlling the 
zoom camera to be at the prescribed position 
and to have the prescribed magnification ratio 
for each of the plurality of image areas obtained 
by the dividing section; and 
a control Information table for storing the preset 
, control information in correspondence with the 
position of each of the plurality of image areas, 

wherein the converting section obtains the 
control information for the image area specified by 
the specifying section from the control information 
table and/or from calculation using a prescribed cal- 
culation procedure. 

7. A composite camera system according to claim 4, 
further comprising: 

an image memory section for temporarily stor- 
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. ing the omnidirectional image taken by the om- 
: nidirectibnal camera frame by frame sequen- 
tially; and 

a moving body-detection section for detecting 
a . movement of a moving body by performing 
.* pattern matching of successive frames of the 
omnidirectional Image stored in the image 
memory section; 

.-. ■ wherein the detected moving body is zoom- 
photographed by the zoom . camera, and an en- 
larged image of the moving body is displayed on the 
display screen of the display section. 



a A composite camera system according to claim 4, 
wherein when the omnidirectional image is circular, 
the dividing section divides the omnidirectional im- 
.. age concentrically and radially, and displays the 
plurality of image areas of the omnidirectional inv 
20 age together with area frames defining the plurality 
of image areas.: 

9. A composite camera system according to claim 6, 
wherein the dividing section displays the piuraiity of 

25 image areas of the omnidirectional image together 
with area frames and area numbers defining the 
plurality of image areas, and the control Information 
setting section presets the control information for 
controlling the zoom camera to be at the prescribed 

so : position and to have the prescribed magnification 
: ratio for each of the plurality of image areas provid- 
ed with the area numbers. 

10. A composite camera system according to claim 4, 
as hjrthercomprisinganinvalidareasettingsectionfor 

presetting, as invalid, an arbitrary area of the omni- 
' directional image which is unstable due to environ- 
mental influence, 

wherein the invalid area setting section inval- 
40 idates the specification of the image area by the 
specifying section. 
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11. A composite camera system according to claim 7, 
further comprising: 

a monitoring area setting section for setting at 
least one monitoring area in the omnidirectional 
. image; and 
ah alarm information output section for output- 
ting alarm information when the moving body 
detecting section detects that the moving body 
has entered the monitoring area. 

wherein the zoom camera zoom-photographs 
the moving body which has entered the monitoring 
area and displays the . photographed image of the 
. moving body on trie display screen of the display 

• section.: • ■■" 
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12. A composite camera system according to claim 1 , 
further comprising a remote controller connected to 
the omnidirectional camera and the zoom camera 
via a wireless or wired communication line, 

wherein an omnidirectional Image^taken by s 
the omnidirectional camera and an Image taken by 
- the zoom camera are transferred to the remote con- 
troller via the communication line. 

13. A composite camera system according to claim 12, 
wherein the remote controller includes at least the 
dividing section, the specifying section, and the 
converting section. 

1 4. A composite camera image display control method, 
comprising the processing steps of: , 

performing positional control and magnification 
ratio control of at least one zoom camera for 
omnidirectional image data centered on a pre- 
scribed portion thereof, the omnidirectional im- 
age data being obtained by an omnidirectional 
camera capable of taking an omnidirectional 
imago over a viewing angle of a maximum of 
360 degrees; and 
displaying an omnidirectional Image taken by 
the omnidirectional camera and a zoom image 
taken by the zoom camera. 

15. A zoom camera control method for performing po- 
. sitional control and magnification ratio control of at 

least one zoom camera: for omnidirectional image 
data centered on a prescribed portion thereof, the 
omnidirectional image data being obtained by an 
omnidirectional camera capable of taking ah omni- 
directional image over a viewing angle of a maxi- 
mum of 360 degrees, the method comprising the 
steps of: 

displaying an omnidirectional image taken by 
the omnidirectional camera; ^ - 
dividing the omnidirectional image displayed on 
a display screen of the . display section into a 
plurality of image areas; 
automatically or manually specifying at least 
. one image area from the plurality of image ar- 
: eas; • ■ 

converting information on the at least one im- 
age area specified by the specifying section in- 
to control information for controlling the zoom 
camera to be at a prescribed position and to 
have a prescribed magnification ratio for pho- 
tographing the image area specified by the 
specifying section; and 
.- controlling the zoom cam era to be at the pre- 
: - scribed position and to have the prescribed 
. magnification ratio for photographing the image 
area specified by the specifying section, based 



on the control irifonnation converted by thecon- 
verting section. ■.. 

16. A control program for causing a computer to exe- 
cute the steps of: 

displaying an omnidirectional image taken by 
the omnidirectional camera; 
dividing the omnidirectional image displayed on 
- a display screen of thedisplay section into a 
plurality of image areas; 
automatically or manually specifying at least 
one image area from the plurality of image ar- 
eas; 

■"'-*.■ converting information on the at least one im- 
age area specified by the specifying section In- 
■ to control information for controlling the zoom 
camera to be at a prescribed position and to 
have a prescribed magnification ralio for pho- 
: tographing the image area specified by the 
; specifying section; and : 
controlling the zoom camera to be at the pre- 
scribed position arid to have the prescribed 
.; magnification ratio for photographing the imago 
area specified by the specifying section, based 
on the control information converted by the con- 
: verting section. 

17. A computer readable recording medium having a 
control program according to claim 1 6 stored there- 

• .. in. ; : • " • '•" - - " ' 

18. A control, program for causing a computer to exe- 
cute the processing steps of a composite camera 
imagedisplay control method according to daim 14. 

19. A computer readable recording medium having a 
control program according to claim 1 8 stored there- 

in. . • 
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